Conventional network performance monitoring techniques fall into two categories: passive and active. Passive techniques involve examination of existing network traffic and include packet and byte counts for packets matching various criteria, and packet length histograms and stateful measurements such as average packet and data rate measurements. Active techniques involve modifying existing packet traffic, or injecting test traffic from one measurement device addressed to another measurement device, and including enough information in the test packets to extract useful measurements. A common example of an active technique is the “ping” program, which sends an ICMP Echo Request to a remote system, and processes the reply. Such packets typically include the sender's time of transmission, and allow the measurement of loss rates, average end-to-end network transit delay, and delay variance (“jitter”).
A major drawback to active measurement techniques is that such techniques consume network bandwidth—often considerable fractions of the link bandwidth, if highly-accurate results are desired over a relatively short timescale. The bandwidth consumed is, thus, not available to actual user traffic—i.e., it is often seen as wasted, except when actual network performance problems are present. Also, congestion is a common source of network performance problems (too much traffic at one or more points in the network), and sending active measurement packets at such times only worsens the congestion. In addition, there is no guarantee that test traffic will be treated by the network like normal user traffic. This might be due to either normal network packet-classification behavior, or deliberate attempts by the network operator to bias performance tests in its favor.
Passive network measurements techniques, thus, may be more desirable since they do not impact the network as a whole and do not consume otherwise usable bandwidth. Passive network measurement techniques also operate on actual user traffic and, thus, give a more accurate picture of the user's experience on the network. Passive techniques, however, are usually limited to measurements taken at a single point, since there has been no easy way to correlate appearances of the same packet at different places in the network. A conventional technique of this sort involves the collection of “packet traces” at multiple points in the network. These “packet traces” include logs of every packet header seen at that point, with an associated time stamp taken from a global clock source (e.g., a GPS receiver). Packet traces require storing about 100–200 bits from every packet, depending on the intended use. On a high bandwidth link, packet traces typically require huge amounts of storage (e.g., gigabyte disk drives), often with high bandwidth interfaces. For example, a 1 Gb/s interface will typically require a 500 Mb/s trace-collection storage device that, in turn, usually requires a special, high performance disk. Also, trace collection storage usually fills up rapidly, and takes a long time to transfer to a central repository (usually over the network itself, thus using a lot of network bandwidth). Therefore, packet traces generally cover only a few seconds to a few minutes of time, and are rarely taken more than a few times per day.
Therefore, there exists a need for systems and methods that can passively monitor network performance characteristics without requiring large amounts of storage and without using excessive amounts of network bandwidth.